Scalable programming and algorithms for data-intensive life science applications.

Cloud computing [1] offers new approaches for scientific computing that leverage the major commercial hardware and software investment in this area. Closely coupled applications are still unclear in clouds as synchronization costs are still higher than on optimized MPI machines. However loosely coupled problems are very important in many fields and can achieve good cloud performance even when pleasingly parallel steps are followed by reduction operations as supported by MapReduce. It appears that many data analysis problems fit the MapReduce paradigm but there is no definitive analysis here. For example analysis of LHC (Large Hadron Collider) data corresponds to a data selection step followed by forming histograms; this naturally corresponds “perfectly” to the MapReduce paradigm. In Life Science, “all-pairs” applications like BLAST can run well with MapReduce but are particularly simple corresponding to “pleasingly parallel” or “map only” structure. Finally there are applications involving steps like the dimension reduction or clustering algorithms illustrated below where pleasing parallel operations (such as alignment and sequence distance computation) and followed by data mining steps involving iterative operations – such as those present in matrix algebra. Such iterative algorithms are the mainstay of large scale scientific computing and are linked directly to data with data assimilation in weather and climate area [2]. Even in the “birthplace” of MapReduce – Information Retrieval – the Page Rank algorithm needs iterative MapReduce. Thus we pose the following questions.